1. Field of the Invention
The present invention relates to an image processing apparatus and method, Head Mounted Display, program, and recording medium, and more specifically relates to technology for correcting chromatic aberration in a configuration in which the position of primary color elements included in each pixel of a display differs for each color.
2. Description of the Related Art
Recently, so-called MR (Mixed Reality) technology in which real and virtual worlds are merged seamlessly and in real time has become known. In one example of MR technology that is known, using a video see-through HMD, an object being observed from the pupil position of a wearer of the HMD is captured with a video camera or the like, and a mixed reality image in which CG is superimposed on that captured image is displayed to the HMD wearer. HMD is an abbreviation for Head Mounted Display, and CG is an abbreviation for Computer Graphics. With a video see-through HMD, an image of an object is captured using a charge-coupled device such as a CCD to obtain digital image data of the object, and an MR image (mixed reality image) in which a CG image is superimposed is displayed to the wearer via a display device such as a liquid crystal display.
Size and weight reduction are desirable for such an HMD worn on the head. However, when an optical system is configured with inexpensive lenses or few lenses, there is a risk that due to lens aberration, it will not be possible to maintain good image quality for the displayed image. For example, due to lens distortion aberration, a barrel or pin-cushion image may be obtained. Also, due to lens magnification chromatic aberration, color bleeding in red, blue, or green may occur at border portions of the object image. Therefore, it is necessary to correct the above sort of image quality reduction of the object image caused by lens aberration.
Systems of correcting various aberration by an optical approach are generally large in size and weight, and therefore, in many cases, are not appropriate for an HMD application in which reduced size and weight are desired. Therefore, the possibility of allowing the adoption of inexpensive lenses and reducing the number of lenses by implementing electronic correction using signal processing also for the optical system used for capture and display is being investigated.
Technology for using signal processing to correct distortion aberration and magnification chromatic aberration among the various types of aberration of the optical system can be broadly classified into three types. A general description of each of these is given below.
The first type is processing to correct distortion aberration and magnification chromatic aberration using address conversion. Address conversion is a technique in which a distorted image is moved to an ideal image location, based on the corresponding relationship of an imaging position obtained with an ideal optical system and an actual imaging position where there are aberration effects in the capture system. There are various types of address conversion, including converting the corresponding relationship with respect to pixel position before and after conversion to a table, and simply converting the corresponding relationship (address) of memory read out and writing, as well as address conversion that maintains coordinate data after high-accuracy conversion. In the display system as well, the display position is converted according to the corresponding relationship of the pixels displayed and the position where they are actually displayed. Distortion aberration can be corrected by performing such pixel conversion, and magnification chromatic aberration can be corrected when conversion has been performed for each color constituting a pixel.
The second type is processing to correct magnification chromatic aberration using resolution conversion, in which an image with little color bleeding is obtained by applying enlargement or reduction processing to a reference color using a variable magnification that differs depending on the color.
The third type is processing to correct distortion aberration using an approximation polynomial and processing to correct magnification chromatic aberration using distortion aberration correction of each color, in which a set of coordinates after conversion by approximation with a higher order polynomial using correction parameters as coefficients are calculated.
Address conversion has comparatively high versatility, and with address conversion, highly accurate coordinate conversion is possible. However, when image size is large or when high conversion accuracy is sought, the size of a look-up table in which the corresponding relationship with coordinates after conversion is stored will become inflated.
In response, a configuration in which one of a plurality of colors is used as a reference color and the difference between the reference color and other colors is stored; and a configuration in which symmetry of an optical system is used to reduce the size of a table, are disclosed in Japanese Patent Laid-Open No. 8-205181 and Japanese Patent Laid-Open No. 2004-153323. For example, if an optical system is symmetrical in the left-right direction or the up-down direction relative to an image, it is possible to generate a look-up value of a display target portion only for a side in one direction, so the table size can be reduced by half. Furthermore, if the optical system is symmetrical in both the left-right direction and the up-down direction (i.e., a rotationally symmetrical system), the table size can be reduced to one quarter.
However, with the conventional technology as described above, the following sort of problems exist. That is, even if the optical system that leads light from the display device to the pupils is rotationally symmetrical or is symmetrical relative to an axis that passes through an optical origin point, the alignment of pixels that constitute the display device may not be symmetrical relative to an axis in the left-right or up-down direction. For example, a display panel such as a single panel TFT (Thin-Film Transistor) liquid crystal panel or an organic EL panel corresponds to this sort of display device. Such display devices are often adopted in an HMD in order to reduce size and weight.
Each pixel of a TFT liquid crystal panel has, for example, filters of three colors R (red), G (green), and B (blue) as primary color elements that emit light in the primary colors. In a TFT liquid crystal panel in which, when viewing the liquid crystal panel from the front, the filters are disposed in each pixel in this order from left to right, relative to a vertical axis that passes through the optical origin point, B is positioned to the outside on the right side of the liquid crystal panel, and R is positioned to the outside on the left side of the liquid crystal panel. When correction values of rotational symmetry or axial symmetry are applied to such a configuration, displacement occurs in the alignment direction of the pixels. The manner of displacement depends on the optical element that is used, and in a configuration in which each pixel has filters of three colors RGB, for example, the displacement amount of R and B is about ⅔ of a pixel in the left-right direction. Ordinarily, the resolution of the optical system of an HMD often has a resolving power of about 1 pixel, and it has been confirmed in testing that color displacement of about 0.1 to 0.3 pixels is visually apparent as false color or color bleeding.
Accordingly, in a case where elements (such as color filters or light emitting units) that emit each primary color are each separately provided within each pixel as in the above-described TFT liquid crystal panel, when optical aberration is corrected by a conventional technique, false color or color bleeding to an extent perceptible to a user will occur. In particular, in an HMD, the display device is used near the pupils of the user, so color displacement within pixels stands out as reduced image quality.